- numerical precision
- 数值精度
English-Chinese computer dictionary (英汉计算机词汇大词典). 2013.
numerical precision
Look at other dictionaries:
Numerical integration — consists of finding numerical approximations for the value S In numerical analysis, numerical integration constitutes a broad family of algorithms for calculating the numerical value of a definite integral, and by extension, the term is also… … Wikipedia
Numerical differentiation — is a technique of numerical analysis to produce an estimate of the derivative of a mathematical function or function subroutine using values from the function and perhaps other knowledge about the function. Contents 1 Finite difference formulae 1 … Wikipedia
Numerical analysis — Babylonian clay tablet BC 7289 (c. 1800–1600 BC) with annotations. The approximation of the square root of 2 is four sexagesimal figures, which is about six decimal figures. 1 + 24/60 + 51/602 + 10/603 = 1.41421296...[1] Numerical analysis is the … Wikipedia
Precision (computer science) — In computer science, precision of a numerical quantity is a measure of the detail in which the quantity is expressed. This is usually measured in bits, but sometimes in decimal digits. It is related to precision in mathematics, which describes… … Wikipedia
numerical analysis — the branch of mathematics dealing with methods for obtaining approximate numerical solutions of mathematical problems. [1925 30] * * * Branch of applied mathematics that studies methods for solving complicated equations using arithmetic… … Universalium
Numerical stability — In the mathematical subfield of numerical analysis, numerical stability is a desirable property of numerical algorithms. The precise definition of stability depends on the context, but it is related to the accuracy of the algorithm. A related… … Wikipedia
Numerical control — CNC redirects here. For other uses, see CNC (disambiguation). A CNC Turning Center … Wikipedia
Numerical sign problem — The numerical sign problem refers to the difficulty of numerically evaluating the integral of a highly oscillatory function of a large number of variables. Numerical methods fail because of the near cancellation of the positive and negative… … Wikipedia
Numerical error — In software engineering and mathematics, numerical error is the combined effect of two kinds of error in a calculation. The first is caused by the finite precision of computations involving floating point or integer values. The second usually… … Wikipedia
Numerical approximations of π — This page is about the history of numerical approximations of the mathematical constant pi;. There is a summarizing table at chronology of computation of pi;. See also history of pi; for other aspects of the evolution of our knowledge about… … Wikipedia
List of numerical analysis topics — This is a list of numerical analysis topics, by Wikipedia page. Contents 1 General 2 Error 3 Elementary and special functions 4 Numerical linear algebra … Wikipedia
